Cache map with sequential tracking for invalidation

ABSTRACT

The described technology is directed towards efficiently invalidating cached data (e.g., expired data) in a hash-mapped cache, e.g., on a timed basis. As a result, data is able returned from the cache without checking for whether that data is expired, (if desired and acceptable), because if expired, the data is only briefly expired since the last invalidation run. To this end, a data structure such as a linked list is maintained to track information representative of hash-mapped cache locations of a hash-mapped cache, in which the information tracks a sequential order of entering data into each hash-mapped cache location. An invalidation run is performed on part of the hash mapped cache, including using the tracking information to invalidate a sequence of one or more cache locations, e.g., only the sequence of those locations that contain expired data.

CROSS-REFERENCE TO RELATED APPLICATION

The present application is a continuation of copending U.S. patentapplication Ser. No. 15/081,812, filed on Mar. 25, 2016.

BACKGROUND

Conventional caches provide a fast way to access data. One typical typeof cache uses a hash function related to a request for data to determinea cache location corresponding to the requested data, which facilitatesconstant time lookup. If the requested data is found in the cache, thedata is returned from the cache. If not, (that is, there is a cachemiss), the data is retrieved from another source (e.g., a physical datastore), returned in response to the request and written into the cacheat the appropriate location.

Each data entry in the cache is typically associated with an expirationvalue, such as a time-to-live or timestamp value. When a request forcached data is received but the data is determined to be expired, theexpired data is typically treated as a cache miss, resulting inreturning the data from the other source and updating the cache entrywith the non-expired data and a new expiration value.

While this expiry mechanism works well for conventional caches likeinternet content and Domain Name System caches, in other scenarios thismechanism may not be appropriate. For example, FIFO (first in, firstout) caches that handle a high volume of requests may operate moreefficiently without checking the expiration value for each request.

SUMMARY

This Summary is provided to introduce a selection of representativeconcepts in a simplified form that are further described below in theDetailed Description. This Summary is not intended to identify keyfeatures or essential features of the claimed subject matter, nor is itintended to be used in any way that would limit the scope of the claimedsubject matter.

Briefly, the technology described herein is directed towards efficientlyinvalidating cached data (e.g., expired data) in a hash-mapped cache,including so that in one or more example implementations, data can bereturned from the cache without checking for whether that data isexpired. One or more aspects are directed towards maintaininginformation representative of hash-mapped cache locations of ahash-mapped cache, in which the information tracks a sequential order ofentering data into each hash-mapped cache location. An invalidation runis performed on at least part of the hash mapped cache, including usingthe information representative of the hash-mapped cache locations toinvalidate a sequence of one or more cache locations.

Other advantages may become apparent from the following detaileddescription when taken in conjunction with the drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

The technology described herein is illustrated by way of example and notlimited in the accompanying figures in which like reference numeralsindicate similar elements and in which:

FIG. 1 is an example representation of components that handle requestsfor data including via a cache that is sequentially tracked forinvalidation, according to one or more example implementations.

FIG. 2 is a block diagram showing an example of updating a cache whilemaintaining a data structure (e.g., a doubly linked list) thatsequentially tracks the order of cached data entries, according to oneor more example implementations.

FIG. 3 is a block diagram showing an example of using a data structure(e.g., a doubly linked list) that sequentially tracks the order ofcached data entries to perform invalidation of cached data, according toone or more example implementations.

FIG. 4 is a block diagram showing an example of using a data structure(e.g., a doubly linked list) that sequentially tracks the order ofcached data entries to perform invalidation of a range of cached data,according to one or more example implementations.

FIG. 5 is a flow diagram showing example steps that may be taken byrequest handler logic to serve cached data when cached or when notcached to obtain the data elsewhere, update the cache and maintain adata structure that sequentially tracks the order of cached dataentries, according to one or more example implementations.

FIG. 6 is a flow diagram showing example steps that may be taken byrequest handler logic to serve cached data when cached and not expired,or when not cached or expired, to obtain the data elsewhere, update thecache and maintain a data structure that sequentially tracks the orderof cached data entries, according to one or more exampleimplementations.

FIG. 7 is a flow diagram showing example steps that may be taken bycache data invalidator logic to invalidate any expired cache dataentries, according to one or more example implementations.

FIG. 8 is a flow diagram showing example steps that may be taken bycache data invalidator logic to invalidate any cache data entries withinan invalidation time window, according to one or more exampleimplementations.

FIG. 9 is a flow diagram showing example steps that may be taken bycache data invalidator logic to invalidate any cache data entries withina certain range of the cache, such as the oldest X percent, according toone or more example implementations.

FIG. 10 is a block diagram representing an example computing environmentinto which aspects of the subject matter described herein may beincorporated.

DETAILED DESCRIPTION

Various aspects of the technology described herein are generallydirected towards separately and occasionally invalidating certain cacheentries based upon invalidation criterion, such as invalidating expiredentries. In general, a separate invalidation process rapidly invalidatesexpired cache entries by only checking for possibly expired entriesrather than traversing the entire cache to find which ones, if any, areexpired. To this end, a data structure such as a doubly linked list isused to track the sequential ordering of cache entry locations that aredispersed (via a hash function) among non-sequential locations in thecache.

During invalidation, the invalidation process, (e.g., which may be runon a configurable and/or best effort basis), may start with the oldestentry, find its location in the cache via the linked list, andinvalidate (e.g., writes NULL into) the entry if expired (or ends if notexpired). If the oldest entry is invalidated, the invalidation processadvances its position in the doubly linked list to sequentially checkeach cache entry in time order for expiration, invaliding each expiredone until a cache entry/location is reached that has not expired, atwhich time the invalidation process ends until run again at some latertime. In one or more implementations, the invalidation may beasynchronous, and in general invalidation does not directly impact thelookup performance (other than to the extent the invalidation processmay share machine resources if the invalidation is run on the samemachine as the cache lookup is performed).

As will be understood, the use of the separate invalidation processavoids having to check the expiration of each request for cached data;the cached data is either present or not (e.g., NULL). At the same time,the invalidation process is fast because the linked list is used to findcache locations, and indeed, is on the order of ten times faster thantraversing the entire cache to invalidate expired entries.

In one or more implementations, the technology described herein thusincludes separately tracking the sequential order of the cached data,and using a doubly linked list of pointers to corresponding cacheentries. To invalidate a time-ordered set of entries in the cache, whichare distributed (pseudo-randomly) throughout the cache according to thehash function, the oldest entry in the linked list is tracked and thelinked list is accessed. Then, the cache pointers in this section of thelinked list are used to quickly locate the cache entries for whichinvalidation is desired.

Note that one or more implementations use a doubly linked list as thetracking data structure, in which along with a forward pointer to thenext node, the nodes have a back pointer to each one's previous node,which supports on-demand random-access cache invalidation, e.g.,explicit invalidation of any cache entry or entries. However,alternative implementations in which such on-demand random-access cacheinvalidation is not appropriate may use a circular, singly linked list.

It should be understood that any of the examples herein arenon-limiting. For instance, the invalidation process is not limited toinvalidating actually expired entries, and for example may invalidateany entries, such as those already expired as well as those which willexpire in X seconds. Thus, invalidation can be performed on any timewindow, e.g., invalidate anything with a TTL (time-to-live) value lessthan Z, or between X and Y and so on. It is also feasible for theinvalidation process to invalidate some amount of cache space based uponnumber of entries (or possibly cache size) rather than based solely on(or even at all on) a TTL or a timestamp value, e.g., invalidate theoldest fifty percent of the cache.

As such, the technology described herein is not limited to anyparticular embodiments, aspects, concepts, structures, functionalitiesor examples described herein. As one example, a FIFO cache is one typeof cache exemplified herein, however the technology is not limited toFIFO caches. Similarly, a (doubly) linked list is one suitable type ofsequential cache entry tracking data structure, but the technology isnot limited to linked lists, as any suitable array/data structure may beused. Rather, any of the embodiments, aspects, concepts, structures,functionalities or examples described herein are non-limiting, and thetechnology may be used in various ways that provide benefits andadvantages in computing and data caching/cache management in general.

FIG. 1 shows an example block diagram in which various aspects of thetechnology described herein may be implemented. In FIG. 1, a request 102for data is received by a request handler 104, e.g., a front endcomponent of an internet service or the like. Note that in general, someof the operations represented in FIG. 1 are identified via labeledarrows; e.g., the request 102 corresponds to labeled arrow one (1).

Using data (e.g., an identifier) in the request, the request handler 104uses a hash function 106 (labeled arrows two (2) and three (3)) to mapto a location in a cache 108 and accesses the cache 108, e.g., using thehash value as an index (labeled arrow four (4)). Labeled arrow five (5)represents the cache returning the data if cached or a cache miss ifdata is not cached at that location.

If the requested data is cached, the request handler 104 returns thedata in a response 110 to the request 102, corresponding to labeledarrow (6); otherwise there is no response returned at this time, as therequested data needs to be obtained elsewhere. Note that the arrowslabeled one (1) through six (6) are solid lines, indicating a first setof example operations that is often performed to fully satisfy arequest.

Alternatively, if the requested data is not in the cache 108, furtherexample operations represented in FIG. 1 correspond to dashed arrowslabeled seven (7) through eleven (11). In general, for data not in thecache, the request handler 104 obtains the data from one or more datasources 112, (which also may include one or more caches), as representedvia labeled arrows seven (7) and eight (8), and returns the data in in aresponse 110 to the request 102 (arrow nine (9)).

As represented by the arrow labeled ten (10), the request handler 104(or other suitable component) writes the data into the cache at thelocation corresponding to the previously computed hash function.Further, an expiration value (e.g., a TTL value or timestamp) istypically written into the cache 108 along with the data in one or moreimplementations.

As described herein, the request handler 104 (or other suitablecomponent) also updates a sequential entry tracking data structure 114,e.g., a linked list, as represented by the arrow labeled eleven (11). Aswill be understood, this data structure 114 tracks the order oflocations to which data is entered into the cache 108, and, for example,comprises an array that for each cache data entry maintains a pointer tothe data entry's corresponding hash-mapped location.

An example of one such data structure 114 is represented as a doublylinked list 214 in FIG. 2. As can be seen, each time data is enteredinto the cache 108, the request handler 104 adds a pointer to the cachelocation at the current linked list sequence array position value 222,and increments the (moves to the next) sequence array position value222. Thus, the cache locations are maintained sequentially in the linkedlist 214, e.g., represented by indices T1-Tzzz. In the example of FIG.1, the linked list is a doubly linked list; however this is only oneexample data structure possibility. Note that the linked list entriesneed not maintain the time of entry into the cache, only the order ofentry. Thus, in the example of FIG. 2, along with the forward and backpointers, one linked list entry T0 points to cache data location c, T1points to cache location h, T2 to cache location a, and so on. However,an alternative is to have the data structure 114 maintain an expirationvalue rather than (or in addition to) having an expiration value in thecache. In any event, there is an expiration value (TTL or timestamp)associated with each cached data entry.

Returning to FIG. 1, a process represented as a cache data invalidator116 is triggered (block 118, arrow labeled (a)) to invalidate a“sequence” of zero or more cached data entries, which in manyimplementations correspond to (but are not limited to) those that areexpired. To this end, the cache data invalidator 116 accesses thesequential entry tracking data structure 114 (arrows (b) and (c)), basedupon data representing the oldest data entry, and using the pointerlocation to that oldest location begins invalidating any expired entries(arrow (d)). Note that the oldest entry may not be expired, in whichevent any more recent entries are also not expired, (assuming animplementation in which expiration TTL values are the same). To be ofmuch use with a cache of any significant number of entries, theinvalidator 116 often will have a fairly large sequence to invalidate.

If expired, the cache entry at the oldest location is invalidated, e.g.,set to NULL, and the process continues with the next sequential entrybased upon the next position in the sequential entry tracking datastructure 114. Invalidation continues for any expired data entry; onceone is reached that is not expired, then the more recent ones are alsonot expired, and the cache data invalidator process ends, after savingthe current position in the tracking data structure 114 for starting thenext invalidation run.

FIG. 3 shows a more specific example, in which the cache datainvalidator 116 starts at the oldest data entry according to the currentposition in the linked list, which in this example is T1; (e.g., T0 wasinvalidated in a previous invalidation run, or the linked list iscircular and T0 is not the oldest position). For example, the cache datainvalidator 116 may maintain this starting position (e.g., a pointer tothe doubly linked list node) in internal object memory tracking datastructure 330, or at some other accessible storage location.

When triggered, the cache data invalidator 116 accesses the linked listto quickly locate those data entries in the cache that are to beinvalidated. Note that triggering may be time based, e.g., aconfigurable periodic value that allows trading off an acceptable levelof data “staleness” versus the consumption of computing resources (e.g.,processing time, particularly in single-threaded applications, such asthose run in the node.js runtime environment, in which requested data isinaccessible during invalidation) to perform the invalidation run. Forexample, in many scenarios, data that is a couple of seconds over itsexpiration time is still quite acceptable to use. Thus, the cache datainvalidator 116 may be triggered to run every couple of seconds. Anysystem employing this technology can be configured to run invalidationas desired, to tradeoff acceptable cached data staleness versus theprocessing needed to perform invalidation.

Notwithstanding, other triggers may be used instead of or in addition toperiodic triggering of the cache data invalidator 116. For example, adatabase (one of possibly many data sources) may have been updated, atwhich time it may make sense to trigger the cache data invalidator 116on demand (possibly with different invalidation criteria, such as toinvalidate actually expired data and also any data that will expirewithin five minutes). Thus, virtually any criterion or criteria,including timing, may be used to trigger the cache data invalidator 116,as well as to modify any timed triggering, e.g., every two secondsduring weekdays, every four seconds on weekends, and/or to dynamicallyadjust the frequency based upon current resource availability such ashow many requests for data are coming in. Explicit invalidation of anycache entry or set of entries also may be accomplished, e.g., based upontiming, and/or one or more criteria independent of timing.

In the example of FIG. 3, the cache data invalidator 116 looks to the T1position in the linked list 214 to determine the pointer to cachelocation h, which in this example is expired based upon its associatedexpiration value (TTL or timestamp). Thus, cache location h isinvalidated, (e.g., marked NULL), as represented by the crossed “X” overthat location in the cache 108.

Continuing with this example, the cache data invalidator 116 moves tothe next current position, T2, obtains the pointer to cache location a,determines that this data has expired, and invalidates cache location a.The example similarly continues with T3, which corresponds to cachelocation yyyy, in which the invalidator 116 determines that thislocation's data has expired, and invalidates cache location yyyy.However, T4's corresponding location, cache location g, has not expired.Thus, the cache data invalidator 116 leaves this data in cache locationg intact, and saves T4 (pointing to cache location g) as its oldestposition for its next run.

Instead of using the expiration value (whether actually expired orwithin a time window based upon the expiration value) as the solecriterion for determining whether to invalidate a cached entry, totalnumber of cache entries also may be used as a factor, e.g., invalidatesome fraction (e.g., percentage) of the entire cache, e.g., based ontotal number of entries. By way of example, consider that a significantchange to a database has occurred, and there are a large number ofrequests interested in that data. For example, at a certain time, thelatest release of a popular television series episode becomes availablefor streaming as identified within a catalog of titles. It is predictedthat many users will request the catalog at that time and shortlythereafter, and thus expiring older data, independent of its actualexpiration time, may be desirable; (it may also be desirable topre-populate the cache to some extent with certain updated data justbefore the release time, e.g., via a simulated request, and/or to holdactual requests until cached).

FIG. 4 shows the invalidation of some percentage of the cache, which inthis example may be fifty percent. Because the linked list has anordered sequence of pointers to the cached data entries, the oldestfifty percent of data entries corresponds to the same percentage oflinked list positions. Thus, it is straightforward to determine astarting position in the linked list (e.g., the oldest entry, T11 inthis example), compute an ending position (starting point offset plusthe number of total positions times fifty percent), and invalidate fromthe starting linked list position to the ending position (shown as “I”in certain cache locations to represent invalidated). Once finished, thenew oldest position is known to be the next one following the lastinvalidated one, (e.g., T11+50% of the number of positions +1). Notethat percentage (or similar number of entries) based invalidation may bedone in divided runs to avoid having too many pending requests for data,e.g., instead of invalidating the oldest fifty percent of the cache in asingle invalidation run, invalidate the oldest ten percent of the cachefor five invalidation runs in a row.

FIG. 5 is a flow diagram showing example operations of the requesthandler when a request is received at step 502. Step 504 representscomputing the hash value to determine a cache location. Step 506evaluates whether or not the requested data is in the cache at thatlocation. If so, step 510 returns the data from the cache at thatlocation in response to the request.

Note that in FIG. 5, the expiration of the cached data entry is notchecked, whereby serving cached data is highly efficient. For example,when dealing with a large number of data requests, a significant amountof processing is needed to check whether each requested cache data entryis expired. However, as described herein, such cached data entries canbe separately invalidated when expired, e.g., every two seconds, andthus when acceptable for a given application to receive expired datawithin that two second timeframe, this expiration check on each of thelarge number of data requests can be avoided. Notwithstanding, asdescribed below with reference to FIG. 6, the technology describedherein also may be used with applications that need an actual expirationcheck for any cached data.

If not found in the cache at step 506, step 512 represents obtaining thedata from a higher-level data source. This may be a higher-level cache,a database, or other suitable data source, for example. Step 514 returnsthe data in response to the request. Step 516 updates the cache of thecomputed cache location. Step 518 adds a pointer to the cache locationat the current linked list position.

Step 520 increments the current linked list position, that is, moves theposition to the next node pointed at from the last, previous node. Inalternative implementations in which the linked list is a circular listand the current linked list position is at the end, step 520 also wrapsthe linked list position to the start of the linked list.

FIG. 6 is a flow diagram showing example operations similar to those ofFIG. 5 of an alternative request handler (or alternative logic withinthe same request handler) when a request is received (step 602). As willbe understood, however, FIG. 6 differs from FIG. 5 via alternative logicthat may be performed to prevent returning expired data, even if onlybriefly expired. For example, using the same cache that is separatelyinvalidated by invalidation logic as described herein, certain datasensitive applications can flag a request (or all requests) as needingto check the actual expiration.

Thus, step 604 represents computing the hash value to determine a cachelocation, and step 606 evaluates whether or not the requested data is inthe cache at that location. If so, unlike returning the data as in FIG.5, step 608 is performed to evaluate the actual expiration valueassociated with the data. If not expired, step 610 returns the data fromthe cache at that location in response to the request.

If not found in the cache at step 606, or found but determined to beexpired at step 608, step 612 represents obtaining the data from ahigher-level data source. This may be a higher-level cache, a database,or other suitable data source, for example. Step 614 returns the data inresponse to the request.

Step 616 updates the cache of the computed cache location. Step 618 addsa pointer to the cache location at the current linked list position.Step 620 increments the current linked list position, that is, moves tothe next node. As with FIG. 5, if in alternative implementations inwhich the linked list is a circular list and the current linked listposition is at the end, step 620 also wraps the linked list position tothe start of the linked list.

FIG. 7 represents example logic of the cache data invalidator when runin an implementation in which a cache entry is invalidated if expired,otherwise the cached data is left intact and the cache data invalidatorprocess ends. Step 702 represents determining the starting position inthe linked list, which in this example is the oldest entry as maintainedin the tracking data structure 330 (FIG. 3).

Step 704 obtains the pointer in the linked list at this position to findthe cache location corresponding to this oldest entry. Step 706evaluates whether the data entry has expired. If so, step 710invalidates the cache entry corresponding to the current linked listposition. Step 712 moves to the next node (e.g., increments the currentlinked list position) and returns to step 704 which uses the cachepointer maintained at this next position to find the cache location forthe next set of data sequentially entered into the cache. Step 706evaluates the cached data's expiration value, invalidating the cacheddata entry if expired. As can be seen from the logic of FIG. 7, theprocess continues until step 706 determines that a cached data entry hasnot expired. Upon such a determination, the invalidation process savesthe current linked list position in the tracking data structure 330, andthe invalidation process ends.

As can be readily appreciated, nodes that point to invalidated cachelocations may be removed from the doubly linked list by manipulating thenodes' pointers. This can be done as each entry is invalidated, e.g., aspart of step 712, or can be done at once, as the starting node (beforebeing removed) has a pointer to the previous valid node and the lastnode (before being removed) has a pointer to the next valid node; theprevious valid node is modified to point to the next valid node, andvice-versa for the back pointer.

FIG. 8 represents example logic of a cache data invalidator when runningin an implementation in which a cached data entry is invalidated ifwithin a time invalidation window. Step 802 represents determining thestarting position in the linked list, which may be the oldest entry asmaintained in the tracking data structure 330 (FIG. 3). However, it isfeasible to start at any position, such as to find a starting pointbased upon a binary search of the expiration values in the cache, e.g.,to invalidate a certain section of the cache having data entered thereinwithin a certain time window.

Step 804 obtains the pointer in the linked list at this position to findthe cache location corresponding to this starting cached data entry.Step 806 evaluates whether the data entry is within the invalidationwindow. For example. If the expiration value is a TTL (and all TTLs arethe same such as 7200 seconds when inserted), then checking theremaining time (e.g., less than 3600 seconds or not) determines whetherthe data entry is within a one-hour invalidation window. If theevaluation value is a timestamp, the time window may be determined bysubtracting from the current time (e.g., if current time minus thecached time is greater than one hour then the data entry was cachedwithin the invalidation window).

If within the invalidation window, step 810 invalidates the cache entrycorresponding to the current linked list position. Step 812 incrementsthe current linked list position and returns to step 804 which uses thepointer maintained at this next position to find the cache location forthe next set of data sequentially entered into the cache. As can be seenfrom the logic of FIG. 8, the process continues until step 806determines that a cached data entry is not within the invalidationwindow. Upon such a determination, the invalidation process saves thecurrent linked list position in the tracking data structure 330, and theinvalidation process ends.

FIG. 9 represents example logic of a cache data invalidator when runningin an implementation in which a cached data entry is invalidated ifwithin some range of data entries, e.g., a percentage of the total cacheentries, regardless of expiration values. Instead of a percentage, someother limit may be used, e.g., invalidate the oldest one-thousand dataentries in the cache. Again, the starting position in the linked listmay be something other than the oldest position, and determined in anymanner, however in this example of FIG. 9 the starting positioncorresponds to the oldest cache entry, e.g., as tracked by the cachedata invalidator logic as a position (node pointer) in the linked list.

Step 906 uses the cache pointer in the linked list at the currentposition to find the cache location, which is invalidated at step 908.Step 910 moves to the next node/increments the current linked listposition, (including wrapping back to the start of the linked list ifappropriate).

Step 912 evaluates whether the current linked list position is past theending position computed at step 904. If not, the process loops back,invalidating until cached data corresponding to the ending position isinvalidated, with the linked list position incremented to the next nodebeyond this ending position.

When past the ending position, the invalidation process saves thecurrent linked list position (e.g., a reference to the node) in thetracking data structure 330, as this position now represents the oldestdata entry, and the invalidation process ends.

As can be seen, the use of a tracking data structure that maintains theorder of entries into a hash-mapped cache provides for rapidinvalidation of a desired set of cache entries, e.g., those that areexpired. As a result, a separate process can efficiently performinvalidation, whereby in many scenarios cached data may be returnedwithout having to perform an expiration check. The invalidation processmay be configured to run as desired, including with fixed or variablerecurring timing based upon any suitable criterion or criteria.

One or more aspects are directed towards maintaining informationrepresentative of hash-mapped cache locations of a hash-mapped cache, inwhich the information tracks a sequential order of entering data intoeach hash-mapped cache location. An invalidation run is performed on atleast part of the hash mapped cache, including using the informationrepresentative of the hash-mapped cache locations to invalidate data inone or more hash-mapped cache locations.

Performing the invalidation run may include using the information toinvalidate a set of cache data entries that are expired. This mayinclude stopping the invalidation run when the information points to ahash-mapped cache location having a cache data entry that is notexpired. Performing the invalidation run may include using theinformation to invalidate a set of cache data entries that are expiredor will be expired within a certain time, using the information toinvalidate a set of cache data entries that are within an expirationtime window and/or using the information to invalidate a set of cachedata entries based upon a number of cache entries or a fraction of totalcache entries. Performing the invalidation run may include triggeringthe invalidation run based upon one or more criteria.

Maintaining the information representative of the cache locations mayinclude maintaining a doubly linked list having a plurality of nodes,including maintaining a node for each cache location having valid data,each node including one node pointer to a next node, a cache pointer toits corresponding cache location in the hash-mapped cache, and anothernode pointer to a previous node. Maintaining the informationrepresentative of the cache locations may include maintaining a linkedlist having a plurality of nodes, including maintaining a node for eachcache location having valid data, each node including a node pointer toa next node and a cache pointer to its corresponding cache location inthe hash-mapped cache.

Performing the invalidation run may include a) selecting a noderepresenting an oldest position in the in the sequential order as acurrently selected node, b) using the cache pointer in the currentlyselected node to find a corresponding cache location, c) determining ifthe cache data entry at the corresponding cache location is expired, andif not, advancing to step f), d) invalidating the cache data entry atthe corresponding cache location, e) using the node pointer in thecurrently selected node to select a next node as the currently selectednode and returning to step b), f) maintaining information of thecurrently selected node to represent the oldest position in the in thesequential order, and ending the invalidation run.

Also described is receiving a request to access data in the hash-mappedcache, performing a hash computation based upon at least part of therequest to determine a location in the hash-mapped cache, and if dataexists in the location, returning the data from the location, withoutchecking for expiration of that data, in response to the request.Alternatively, upon receiving a request to access data in thehash-mapped cache, described is performing a hash computation based uponat least part of the request to determine a location in the hash-mappedcache, and if data exists in the location and is not expired, returningthe data from the location in response to the request.

One or more aspects are directed towards a hash-mapped data cachecontaining data entries, a tracking data structure that maintains anorder of locations of the data entries as entered into the hash-mappeddata cache, and an invalidator, the invalidator configured to access thetracking data structure to invalidate a set of one or more data entriesbased upon an expiration value associated with each data entry and theorder that the data entries were entered into the cache.

The tracking data structure may comprise a doubly linked list of nodeshaving cache pointers to locations of the data entries in the hashmapped data cache, or alternatively may comprise a singly linked list ofnodes having cache pointers to locations of the data entries in the hashmapped data cache.

The hash-mapped data cache may comprise an in-memory cache. A requesthandler may request data from the hash-mapped data cache, including datathat is expired based on an associated expiration value but has not yetinvalidated by the invalidator.

The invalidator may comprise an asynchronous process. The invalidatormay be triggered at a rate that is selectable so as to not impactperformance of data lookup in the hash-mapped data cache or to reduceany impact of the performance of the data lookup in the hash-mapped datacache.

One or more aspects are directed towards a) maintaining a hash-mappedcache with locations for cached data, b) maintaining a linked listhaving a plurality of nodes, including maintaining a node for each cachelocation having valid data, each node including a node pointer to a nextnode and a cache pointer to its corresponding cache location in thehash-mapped cache, in which the linked list node order corresponds to anorder of entering data into the hash-mapped cache locations, c)performing an invalidation run, including selecting a starting node as acurrently selected node. d) using the cache pointer in the currentlyselected node to find a corresponding cache location, e) determining ifthe cache data entry at the corresponding cache location is to beinvalidated, and if not, advancing to step h), f) invalidating the cachedata entry at the corresponding cache location, g) using the nodepointer in the currently selected node to select a next node as thecurrently selected node and returning to step d), h) ending theinvalidation run.

Determining if the cache data entry at the corresponding cache locationis to be invalidated may include determining if the cache data entry isexpired based on an expiration value associated therewith. Determiningif the cache data entry at the corresponding cache location is to beinvalidated may include determining if the cache data entry is a)expired or soon to be expired based on an expiration value associatedtherewith, b) corresponds to data entered within a specified timewindow, c) based upon a number of cache entries or d) based upon afraction of total cache entries.

Performing the invalidation run may comprise triggering the invalidationrun based upon one or more criteria.

Further described is handling requests for data from the cache,independent of the invalidation run, including returning data from thecache that is expired based on an associated expiration value but hasnot yet invalidated by the invalidator.

Example Computing Device

The techniques described herein can be applied to any device or set ofdevices (machines) capable of running programs and processes. It can beunderstood, therefore, that personal computers, laptops, handheld,portable and other computing devices and computing objects of all kindsincluding cell phones, tablet/slate computers, gaming/entertainmentconsoles and the like are contemplated for use in connection withvarious implementations including those exemplified herein. Accordingly,the general purpose computing mechanism described below in FIG. 10 isbut one example of a computing device.

Implementations can partly be implemented via an operating system, foruse by a developer of services for a device or object, and/or includedwithin application software that operates to perform one or morefunctional aspects of the various implementations described herein.Software may be described in the general context of computer executableinstructions, such as program modules, being executed by one or morecomputers, such as client workstations, servers or other devices. Thoseskilled in the art will appreciate that computer systems have a varietyof configurations and protocols that can be used to communicate data,and thus, no particular configuration or protocol is consideredlimiting.

FIG. 10 thus illustrates an example of a suitable computing systemenvironment 1000 in which one or aspects of the implementationsdescribed herein can be implemented, although as made clear above, thecomputing system environment 1000 is only one example of a suitablecomputing environment and is not intended to suggest any limitation asto scope of use or functionality. In addition, the computing systemenvironment 1000 is not intended to be interpreted as having anydependency relating to any one or combination of components illustratedin the example computing system environment 1000.

With reference to FIG. 10, an example device for implementing one ormore implementations includes a general purpose computing device in theform of a computer 1010. Components of computer 1010 may include, butare not limited to, a processing unit 1020, a system memory 1030, and asystem bus 1022 that couples various system components including thesystem memory to the processing unit 1020.

Computer 1010 typically includes a variety of machine (e.g., computer)readable media and can be any available media that can be accessed by amachine such as the computer 1010. The system memory 1030 may includecomputer storage media in the form of volatile and/or nonvolatile memorysuch as read only memory (ROM) and/or random access memory (RAM), andhard drive media, optical storage media, flash media, and so forth. Byway of example, and not limitation, system memory 1030 may also includean operating system, application programs, other program modules, andprogram data.

A user can enter commands and information into the computer 1010 throughone or more input devices 1040. A monitor or other type of displaydevice is also connected to the system bus 1022 via an interface, suchas output interface 1050. In addition to a monitor, computers can alsoinclude other peripheral output devices such as speakers and a printer,which may be connected through output interface 1050.

The computer 1010 may operate in a networked or distributed environmentusing logical connections to one or more other remote computers, such asremote computer 1070. The remote computer 1070 may be a personalcomputer, a server, a router, a network PC, a peer device or othercommon network node, or any other remote media consumption ortransmission device, and may include any or all of the elementsdescribed above relative to the computer 1010. The logical connectionsdepicted in FIG. 10 include a network 1072, such as a local area network(LAN) or a wide area network (WAN), but may also include othernetworks/buses. Such networking environments are commonplace in homes,offices, enterprise-wide computer networks, intranets and the Internet.

As mentioned above, while example implementations have been described inconnection with various computing devices and network architectures, theunderlying concepts may be applied to any network system and anycomputing device or system in which it is desirable to implement suchtechnology.

Also, there are multiple ways to implement the same or similarfunctionality, e.g., an appropriate API, tool kit, driver code,operating system, control, standalone or downloadable software object,etc., which enables applications and services to take advantage of thetechniques provided herein. Thus, implementations herein arecontemplated from the standpoint of an API (or other software object),as well as from a software or hardware object that implements one ormore implementations as described herein. Thus, various implementationsdescribed herein can have aspects that are wholly in hardware, partly inhardware and partly in software, as well as wholly in software.

The word “example” is used herein to mean serving as an example,instance, or illustration. For the avoidance of doubt, the subjectmatter disclosed herein is not limited by such examples. In addition,any aspect or design described herein as “example” is not necessarily tobe construed as preferred or advantageous over other aspects or designs,nor is it meant to preclude equivalent example structures and techniquesknown to those of ordinary skill in the art. Furthermore, to the extentthat the terms “includes,” “has,” “contains,” and other similar wordsare used, for the avoidance of doubt, such terms are intended to beinclusive in a manner similar to the term “comprising” as an opentransition word without precluding any additional or other elements whenemployed in a claim.

As mentioned, the various techniques described herein may be implementedin connection with hardware or software or, where appropriate, with acombination of both. As used herein, the terms “component,” “module,”“system” and the like are likewise intended to refer to acomputer-related entity, either hardware, a combination of hardware andsoftware, software, or software in execution. For example, a componentmay be, but is not limited to being, a process running on a processor, aprocessor, an object, an executable, a thread of execution, a program,and/or a computer. By way of illustration, both an application runningon a computer and the computer can be a component. One or morecomponents may reside within a process and/or thread of execution and acomponent may be localized on one computer and/or distributed betweentwo or more computers.

The aforementioned systems have been described with respect tointeraction between several components. It can be appreciated that suchsystems and components can include those components or specifiedsub-components, some of the specified components or sub-components,and/or additional components, and according to various permutations andcombinations of the foregoing. Sub-components can also be implemented ascomponents communicatively coupled to other components rather thanincluded within parent components (hierarchical). Additionally, it canbe noted that one or more components may be combined into a singlecomponent providing aggregate functionality or divided into severalseparate sub-components, and that any one or more middle layers, such asa management layer, may be provided to communicatively couple to suchsub-components in order to provide integrated functionality. Anycomponents described herein may also interact with one or more othercomponents not specifically described herein but generally known bythose of skill in the art.

In view of the example systems described herein, methodologies that maybe implemented in accordance with the described subject matter can alsobe appreciated with reference to the flowcharts/flow diagrams of thevarious figures. While for purposes of simplicity of explanation, themethodologies are shown and described as a series of blocks, it is to beunderstood and appreciated that the various implementations are notlimited by the order of the blocks, as some blocks may occur indifferent orders and/or concurrently with other blocks from what isdepicted and described herein. Where non-sequential, or branched, flowis illustrated via flowcharts/flow diagrams, it can be appreciated thatvarious other branches, flow paths, and orders of the blocks, may beimplemented which achieve the same or a similar result. Moreover, someillustrated blocks are optional in implementing the methodologiesdescribed herein.

CONCLUSION

While the invention is susceptible to various modifications andalternative constructions, certain illustrated implementations thereofare shown in the drawings and have been described above in detail. Itshould be understood, however, that there is no intention to limit theinvention to the specific forms disclosed, but on the contrary, theintention is to cover all modifications, alternative constructions, andequivalents falling within the spirit and scope of the invention.

In addition to the various implementations described herein, it is to beunderstood that other similar implementations can be used ormodifications and additions can be made to the describedimplementation(s) for performing the same or equivalent function of thecorresponding implementation(s) without deviating therefrom. Stillfurther, multiple processing chips or multiple devices can share theperformance of one or more functions described herein, and similarly,storage can be effected across a plurality of devices. Accordingly, theinvention is not to be limited to any single implementation, but ratheris to be construed in breadth, spirit and scope in accordance with theappended claims.

What is claimed is:
 1. A method comprising: maintaining informationrepresentative of hash-mapped cache locations of a hash-mapped cache, inwhich the information tracks a sequential order of entering data intothe hash-mapped cache locations; performing an invalidation run on atleast part of the hash mapped cache, including using the informationrepresentative of the hash-mapped cache locations to invalidate datacorresponding to oldest data, in the one or more hash-mapped cachelocations, based on the sequential order; receiving a request to accessrequested data in the hash-mapped cache; performing a hash computationbased upon at least part of the request to determine a location in thehash-mapped cache; and in response to determining that the requesteddata exists in the location and is not invalidated, returning therequested data from the location in response to the request withoutchecking whether the requested data is expired.
 2. The method of claim 1wherein performing the invalidation run comprises invalidating a set ofcache data entries that are expired.
 3. The method of claim 1 whereinperforming the invalidation run comprises invalidating at least onecache data entry that is expired and at least one cache data entry thatis about to expire.
 4. The method of claim 1 wherein performing theinvalidation run comprises invalidating a percentage of the cachecorresponding to oldest data based on the sequential order.
 5. Themethod of claim 1 wherein performing the invalidation run comprisesinvalidating a cache data entry that is expired based on a timestampassociated with the cache data entry.
 6. The method of claim 1 whereinperforming the invalidation run comprises invalidating a cache dataentry that is expired based on a time-to-live value associated with thecache data entry.
 7. The method of claim 1, wherein performing theinvalidation run comprises triggering the invalidation run based uponone or more criteria.
 8. A system comprising: a hash-mapped data cachecontaining data entries; a tracking data structure that maintains anorder of locations of the data entries as entered into the hash-mappeddata cache; an invalidator, the invalidator configured to access thetracking data structure to invalidate a set of one or more data entriesbased upon a time-based expiration value associated with each data entryand the order that the data entries were entered into the cache; and arequest handling process that: receives a request to access requesteddata in the hash-mapped cache, performs a hash computation based upon atleast part of the request to determine a location in the hash-mappedcache, evaluates whether the requested data exists in the location andis not invalidated, and if the requested data exists in the location andis not invalidated, returns the requested data from the location inresponse to the request without checking whether the requested data isexpired.
 9. The system of claim 8 wherein the tracking data structurecomprises a doubly linked list of nodes having cache pointers tolocations of the data entries in the hash mapped data cache.
 10. Thesystem of claim 8 wherein the invalidator comprises an asynchronousprocess.
 11. The system of claim 8 wherein at least one time-basedexpiration value comprises a timestamp.
 12. The system of claim 8wherein at least one time-based expiration value comprises atime-to-live value.
 13. The system of claim 8, wherein the tracking datastructure comprises a singly linked list of nodes having cache pointersto locations of the data entries in the hash mapped data cache.
 14. Oneor more machine-readable storage media having machine-executableinstructions, which when executed perform operations, the operationscomprising: a) maintaining a hash-mapped cache with locations for cacheddata; b) maintaining a linked list having a plurality of nodes,including maintaining a node for each cache location having valid data,each node including a node pointer to a next node and a cache pointer toits corresponding cache location in the hash-mapped cache, in which thelinked list node order corresponds to an order of entering data into thehash-mapped cache locations; c) performing an invalidation run,including selecting a starting node as a currently selected node; d)using the cache pointer in the currently selected node to find acorresponding cache location; e) determining if the cache data entry atthe corresponding cache location is to be invalidated, invalidated, andif not, advancing to step h); f) invalidating the cache data entry atthe corresponding cache location, g) using the node pointer in thecurrently selected node to select a next node as the currently selectednode and returning to step d); h) ending the invalidation run; i)receiving a request to access requested data in the hash-mapped cache;j) performing a hash computation based upon at least part of the requestto determine a location in the hash-mapped cache; and k) in response todetermining that the requested data exists in the location and is notinvalidated, returning the requested data from the location in responseto the request without checking whether the requested data is expired.15. The one or more machine-readable storage media of claim 14 whereindetermining if the cache data entry at the corresponding cache locationis to be invalidated comprises determining if the cache data entry isexpired based on a timestamp associated therewith.
 16. The one or moremachine-readable storage media of claim 14 wherein determining if thecache data entry at the corresponding cache location is to beinvalidated comprises determining if the cache data entry is expiredbased on a time-to-live value associated therewith.
 17. The one or moremachine-readable storage media of claim 14 wherein performing theinvalidation run comprises invalidating at least one cache data entrythat is about to expire based on a time-based expiration valueassociated therewith.
 18. The one or more machine-readable storage mediaof claim 14, wherein the request comprises a first request and requesteddata comprises first requested data, and wherein the instructionsfurther comprise receiving a second request to access second requesteddata in the hash-mapped cache, performing a hash computation based uponat least part of the second request to determine a location in thehash-mapped cache, and in response to determining that the secondrequested data exists in the location and is not invalidated, returningthe second requested data from the location in response to the request.19. The one or more machine-readable storage media of claim 14 whereinselecting the starting node as the currently selected node comprisesselecting the oldest node based on the order of entering the data intothe hash-mapped cache locations.
 20. The one or more machine-readablestorage media of claim 14, wherein the operations further comprisetriggering the invalidation run based upon one or more criteria.